Example 1. Any equation can be converted into the above form with an appropriate F. For example, for the equation \cos x = \exp x + 2 we can take F(x) := \cos x - \exp x - 2 or F(x) := \exp x + 2 - \cos x, or even F(x) = \frac 1{\cos x - \exp x -1} - 1 (but we need to be a bit careful about the domain of this function).

Example 2. A transcendental equation \exp x + x = 0 has a solution x = - W(1) \approx -0.567143 according to Wolfram Alpha. Here W is the product log function. How do we compute W(1)? Well, it’s not easy.

For equation \cos x = x\qquad (F(x) = \cos x - x \text{ for example}) Wolfram Alpha only gives an approximate numerical value.

Cubic equations like x^3 - x - 1 = 0 can be solved using various formulas (see the solution on Wolfram Alpha), but the formulas are very complicated in general.

Example 3. Consider the quadratic equation x^2 - 1 = 0. It is clear that the equation has 2 roots, x = 1 and x = -1. Let us use Newton’s method to find one of these roots.

We can take F(x) = x^2 - 1 and F'(x) = 2x. Therefore the iterative formula for Newton’s method is

x_{k+1} = x_k - \frac{x_k^2 - 1}{2x_k}

Here is a sample code in Fortran. We do 5 iterations of the Newton’s method.

! Implementation of Newton's method for x^2 - 1 = 0
implicit none
real x, F, Fprime
integer i

! Enter the starting x0
read *, x
print *, x

do i = 1, 5 
    ! To solve a different equation, we just need to modify these formulas
    F = x**2 - 1
    Fprime = 2 * x
    ! Fprime cannot be zero (or too small, there might be a problem)
    if (abs(Fprime) < 1e-6) then
        print *, 'Error: the derivative is too close to zero'
        stop
    endif
    x = x - F / Fprime
    print *, x
enddo

end

With initial guess 2, the method converges to the exact solution very fast:

$ ./a.exe <<< 2
   2.00000000    
   1.25000000    
   1.02499998    
   1.00030482    
   1.00000000    
   1.00000000    

You can see that the number of correct digits approximately doubles in each iteration.

What if we start close to 0?

$ ./a.exe <<< 0.001
   1.00000005E-03
   500.000488    
   250.001251    
   125.002625    
   62.5053101    
   31.2606544    

In this case we first go very far from the correct solution and then converge relatively slowly. Do you see why?

It is therefore very important to choose a good starting point.

For a negative starting value, we end up at -1:

$ ./a.exe <<< -0.5
 -0.500000000    
  -1.25000000    
  -1.02499998    
  -1.00030482    
  -1.00000000    
  -1.00000000    

And if we choose wrong, we get an error:

$ ./a.exe <<< 0
   0.00000000    
 Error: the derivative is too close to zero

Assuming that the x_k converges to the root 1, let us see how the error {\varepsilon}_k := |x_k - 1| of the method behaves. First we subtract 1 from both sides of Newton’s method formula: x_{k+1} - 1 = x_k - 1 - \frac{x_k^2 - 1}{2x_k} and using x_k^2 - 1 = (x_k - 1)(x_k + 1) we get x_{k+1} - 1 = (x_k - 1)\left(1 - \frac{x_k + 1}{2x_k}\right) = (x_k-1)\frac{2x_k - x_k - 1}{2x_k} = \frac{(x_k - 1)^2}{2x_k}. Taking the absolute value of both sides, we have {\varepsilon}_{k+1} = \frac 1{2|x_k|} {\varepsilon}_k^2. So if we take x_0 \in (0.5, 1.5), we see that {\varepsilon}_{k+1} \leq {\varepsilon}_k^2 and the number of zeros after the decimal point of {\varepsilon}_k at least doubles in each iteration.

Example 4. For a \neq 0, consider the equation ax = 1. The solution is obviously x = 1/a, but let us see what we get from Newton’s method for a specific choice of F.

Taking F(x) = \frac 1x - a, we have F'(x) = -\frac 1{x^2} and so Newton’s method iteration reads x_{k+1} = x_k - \frac{\frac 1{x_k} - a}{-\frac 1{x_k^2}} = 2x_k - ax_k^2. There are no divisions in the formula. We can compute 1/a using only subtraction and multiplication. This is sometimes quite useful. Unfortunately, the initial guess x_0 needs to be quite good for fast convergence.